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ABSTRACT 

The National Center for Education Statistics 
supported this conference as a first step in a dialogue emiong 
forecasters to iDenefit all in the field. Fifty forecasters 
representing 20 different federal aoencies attended the day-long 
session. OlDjectives of the conference include: (1) to bring 
forecasters together for a forum to share information cn data quality 
and sources, forecasting methods, and 'orecast performance; (2) to 
build a Federca Forecasters Network; and (3) to establish a directory 
of FedercG. Forecasters. The keynote address is presented: "Solutions 
to Three Forecasting Problems'^ (J. Scott Armstrong). The three 
problems discussed include: predicting the actions of key 
decision-mctkers, bias in forecasts, and gaiining acceptance of the 
forecast. Work sessions presented include: (1) Topic A — "Data 
Problems" (William Sonnenberg) ; (2) Topic B — "Forecasting 
Methodology" (Paul J. Horn) ; and (3) Topic c — "Forecast Evaluation" 
(Debra Gerald and William Hussar) . The conference agenda, a listing 
of paurticipants, and a bibliographic listing of 64 federal 
forecasting publications are appended. (SLD) 
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FOREWORD 



Through its. activities, the National Center for Education Statistics of the U.S. 
Department of Education recognized that there was a need to bring together Federal 
Government forecasters to discuss issues in the field. While individuals have 
informally discussed data and methods, there has been no formal network that allowed 
forecasters from many Federal agencies to share information. 

The Federal Forecasters Conference held on April 25, 1988 in Washington D.C., 
represents an important first step in organizing Federal Government employees who 
are involved in forecasting in the social, engineering, and behavioral sciences. 
The conference was designed to provide a forum to share information on data quality 
and sources, forecasting methods, and forecast performance. 

Fifty forecasters representing 20 different Federal agencies attended the 
day-long session. The program included opening remarks by Office of Educational 
Research and Improvement and National Center for Education Statistics officials, 
work sessions,, and presentations by experts in the field of forecasting from the 
private sector. 

The following people are recognized for their participation in support of the 
first Federal Forecasters Conference: 

William Sonnenberg of the National Center for Education Statistics for chairing 
the conference; 

Bruno Manno of the Office of Educational Research and Improvement for his 
opening remarks; 

Charles D, Cowan u*' the- National Center for Education Statistics for his 
remarks; 

J. Scott Armstrong of the University of Pennsylvania for his presentation; 

Fred Collopy of the University of Pennsylvania for his demonstration; and 

the discussion leaders and other forecasters from many Federal agencies for 
participating in the conference and sharing their knowledge about forecasting. 

The assistance of many NCES staff is also gratefully acknowledged. Lisa 
Avallone, Debra Gerald, and William Sonnenberg were responsible for planning and 
organizing the conference. Debra Gerald also served as moderator for two of the 
sessions on forecast evaluation. William Sonnenberg moderated the sessions on data 
problems. Paul J. Horn moderated the sessions on forecasting methods, developed 
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the method for rotating the participants through the uifferent sessions, and 
produced graphics. William Hussar was the moderator for a session on forecast 
evaluation and prepared the exhibit materials, while Celeste Loar also produced 
graphics for the conference. Lisa Avallone, Patricia Brown, Henry Gordon, an 
Charlcne Hoffman monitored the exhibit of forecasting publications. Lisa Avallone 
and Celestine Davis prepared the conference materials. Paul R. Hall, Thomas Snyder, 
and Leo J. Eiden advised and supported the activities of the conference. 

The efforts of these people and the conference participants made the forum a 
success and provide a model for future cooperation among Federal forecasters. 



Emerson J. Elliott 

Acting Commissioner, 

National Center for Education Statistics 
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FORECASTERS CAN LEARN FROM EACH OTHER 



I would like to welcome you on behalf of Chester E. Finn, Jr., the Assistant 
Secretary of the Office of Educational Research and Improvement. The Assistant 
Secretary is pleased that the Center has arranged this first-time event. Under the 
leadership of Emerson Elliott, the Center has established new standards, developed 
additional reports, and improved the timeliness of reports. Problems dealing with 
the quantity, quality, and timeliness of education data are being solved. 

The Center has been involved in forecasting for some time. During that time, it 
has developed national projections for elementary and secondary and higher 
education. These projections have been published in Trends in Education, Targeted 
Forecasts, and Projections of Education Statistics. The Center is developing 
State models and, as more data become available, trends on private elementary and ' 
secondary school statistics will be published. 

Forecasters have much to learn from each other. The Center recognizes that 
forecasting is important, because it has an impact on policymaking. This forum will 
provide an excellent opportunity to share information on data quality and sources, 
forecasting methods, and forecast performance. In addition, this cooperation will 
strengthen forecasting on a variety of issues and will lead to more long-term 
discussion in this area and to better informed policymaking. 



Bruno Manno 
Chief of Staff 

Office of Educational Research and 
Improvement 
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THREE CONFERENCE OBJECTIVES 



The National Center for Education Statistics has initiated a number of outreach 
activities concerned with repairing its data bases through the redesign of surveys 
and applying the theory and methods of statistics to survey and sampling problems. 
While we should be mindful of data that are needed to address policy issues, we 
should equally be concerned with the technical aspects of forecasting. This meeting 
represents the first step in establishing a dialogue among forecasters that will 
benefit all of us. 

There are three major objectives for this conference^ The first objective is to 
bring forecasters together by providing a forum where forecasters from different 
agencies and disciplines can meet and share information on forecasting issues. 

The next objective is to build a Federal Forecasters Network through which 
experts in the field will be able to discuss various forecasting problems. This can 
be done informally through increased contacts among individuals and formally through 
meetings such as this one. 

The third objective is to establish a directory of Federal Forecasters. The 
forecasters directory, developed for this conference, lists more th^.n 60 individuals 
involved in the profession. This is a notable beginning. But participants should 
work together to expand the number. 

I urge you to meet forecasters from other agencies and share information on data 
sources^ alternative forecasting techniques, and methods for measuring forecast 
performance. 



Charles D. Cowan 
Chief Statistician 

National Center for Education Statistics 
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FEATURE PRESENTATION 



SOLUTIONS TO THREE FORECASTING PROBLEMS 



by J. Scott Armstrong 

There are three problems that affect forecasting in general, but are of 
particular relevance to Federal forecasters. I selected these three problems 
because they are important and because research has yielded some solutions. 
President Eisenhower once said, "Things are more like they are now than they ever 
were before/ 



Three Problems 

The three problems are: 

1. Predicting the actions of key decisionmakers. 

Forecasts are dependent upon the actions of key people. It is difficult to know 
what these people will do and when they will do it. For example, the U.S. Congress 
considers new economic sanctions to force the South African government to eliminate 
apartheid. How '"ould the Afrikaners respond? How would the business com:^nunity in 
South Africa respond? 

2. Bias in forecasts. 

Decisionmakers tend to use forecasts to support the decisions that they favor. 
Sometimes this is intentional and sometimes it is unintentional. To what extent are 
the forecasts influenced by this bias? How can these political influences be 
overcome? For example, a new minimum wage law is proposed in an effort to improve 
the living conditions of the disadvantaged. Congressman X favors a higher minimum. 
He asks his staff to obtain a forecast of the outcome. 

3. Gaining acceptance of the forecast. 

How can the forecaster gain acceptance of the forecast? This is a problem 
especially when the forecast is unpleasant and may call for change. For example, 
you receive a forecast that the legalization of drugs will greatly reduce crime, 
reduce drug dependency, increase the effectiveness A the judicial system, increase 
the GNP, and increase tax revenues. How will you react? 



Dr. Armstrong is an Associate Professor at the Wharton School at the University 
of Pen^isylvania. He is currently the editor of the International Journal of 
Forecasting and author of the book Long-Range Forecasting: From Crystal Ball to 
Computer. 



If wc had methods to solve the above problems, how could we effectively 
communicate the solutions? If communicated, how can we be sure that people will 
agree to use them? If they agree, will they use ihe method properly? You might ask 
what it would take for you to adopt one of these methods. 

As knowledge is gained, how can the information be used to continually improve 
the forecasting system? How can the knowledge be applied when one is faced with the 
need to make a large number of forecasts in a short period of time? 

The traditional solution for the implementation problem has been to improve our 
communication to the forecaster. Despite numerous shortcomings, this has provided 
benefits. Nevertheless, these benefits are probably not widespread. 

An alternative solution is to ignore the forecaster. Instead, decision support 
systems would be provided directly to the decisionmaker. These systems would use 
inputs from the decisionmaker to create a forecasting model for the situation, make 
the forecasts, and show how these forecasts relate to decisionmaking. 

The availability of high speed personal computers and the development of expert 
systems have converged to enhance the implementation of new methods of forecasting. 
Microcomputers can now analyze large data files and produce forecasts rapidly and 
inexpensively. 

The term "experts* >as been defined in various ways. Expert systems for 

forecasting include * ures for translatingjudgmental forecasting into an 

explicit, reproducib.' that is independent of the expert. This includes the 

formulation of the probicu., the selection of the proper forecasting method, and the 
use of the forecast in decisionmaking. 

The benefits of expert systems are that they 1) help to identify the rules that 
forecasters are using in making forecasts; 2) produce improved rules; 3) apply the 
rules consistently; and 4) save money in situations that require many forecasts. 
Identifying the rules may reveal improper rules or expose unintended biases. For 
example, in trying to forecast which applicants would make the most successful 
employees, the development of an expert system m?.y reveal that irrelevant factors 
such as height, sex, looks, accent, and education are being used. The formal 
encoding of the rules can show how each variable affects the forecast. 

In producing improved rules, the expert system can draw not only on the 
forecaster's knowledge, but also on empirical evidence gained from this and similar 
situations or on findings reported in the literature. 

Consistency provides a safeguard against biases. Using the expert system, the 
rules can be applied more consistently than they can be by the forecaster. The 
results are also more accurate even if the expert system relies only on the 
forecaster's rules. 

The expert system can be used to structure the forecasting problem, that is to 
determine what decisions will be affected, what variables need to be forecasted, 
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what data arc relevant, and how to structure the equations. The user provides 
various inputs and then the system recommends a problem structure, explaining why 
this is being done, even to the extent of providing literature references. 

Once the problem has been structured, the expert system selects the appropriate 
forecasting techniques. These techniques would then be used to obtain the 
forecasts. This method of obtaining forecasts has already received some attention. 
For example, it has been found that the automatic Box-Jenkins models provide 
forecasts that are as good as those made by the leading Box-Jenkins experts. 

The expert system for forecasting can then be linked directly to the planning 
and decisionmaking routines. This greatly reduces the likelihood of biases that 
might be introduced by the forecaster or by the decisionmaker, for that matter. 

To date, the use of expert systems in forecasting has been confined primarily to 
the selection of an appropriate forecasting technique, given a well-structured 
problem and some inputs about the types of methods. Expert systems have neither 
addressed the structuring of the problem nor the use of the forecasts in 
decisionmaking. 

Predicting Actions by Key Decisionmakers 

One area that is of particular interest to government forecasters is how to 
predict the actions and reactions of decisionmakers in conflict. Consider the 
previously mentioned example about what v/ould happen if the United States used new 
economic sanctions against South Africa? Or, whAt the response of various interest 
groups would be if the United States took action in Panama? Or, will country X 
devalue its currency? Or, how would people respond to a proposed major change in 
the tax law? 

Role-playing is an especially promising technique for forecasting how 
decisionmakers in conflict would respond. Role-playing is valuable because it 
provides better insight into an opponent's thinking. 

Bias in Forecasts 

Agreement on good standards of forecasting is also useful in preventing or 
detecting bias. The use of such standards seem to be more important in government 
forecasting than in forecasting for the profit-oriented sector. 

The problems of bias seem to be much larger in scope in the public sector. For 
example, does the government use adequate procedures to eliminate bias in 
forecasting the outcome of mass transportation projects, nuclear power plants, or 
synthetic fuels? An expert system could be used to pose proper questions for the 
forecasting situation. It would then identify what areas might be contrary to 
proper procedure, and what techniques can be used to overcome these problems. The 
system would utilize current expert opinion about the best practice as well as the 
latest research findings. 
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Gaining Acceptance of the Forecast 

Forecasts on important topics meet resistance. Consider the following: Would 
an accurate forecast have been accepted by the U.S. Supreme Court in Brown v. Board 
of Education? That is, would it have affected the decision that led to school 
busing? Stephan (1978), in a foUowup of the predictions provided to the Supreme 
Court by social scientists, found that many of the forecasts that were used to 
support the decision were inaccurate. (He claimed to be among those who had made 
poor forecasts.) One forecast was that school integration would reduce racial 
prejudice. That did not occur. 

One way to gain acceptance of forecasts is to use scenarios. The scenario 
involves converting the forecast into a story about the future. Recent research 
yields suggestions on how to write scenarios in order to gain acceptance of unusual 
or threatening forecasts. The use of vivid examples in the scenario is one way to 
attract attention. Another is to ask decisionmakers to write scenarios in which 
they are participants. Such approaches are likely to lead decisionmakers to take 
unusual forecasts more seriously. 

An expert system could help a forecast gain acceptance by linking the forecast 
directly to the decision. Forecast simulations could be run to show exactly how the 
forecasts have an impact on costs and benefits when different rules are used. 
Outcomes with and without human intervention could also be compared. By asking for 
prior judgments and expected courses of action, the expert system can help identify 
forecasts that would be surprising. The system would then be used to help 
decisionmakers decide what forecast would lead to action and what the potential 
actions that might be taken would be. 

Many of the solutions to the three problems are simple. Despite this, they are 
often ignored. The use of an expert system could lead to direct use of good 
forecasting procedures by decisionmakers. These decisionmakers may then say, 
"Goodbye forecaster." 
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Topic A: DATA PROBLEMS 



Moderator: 



William Sonnenberg, National Center for Education Statistics 



Discussion Leaders: 



Ted Van Hintum, Veterans* Administration 
Russ Geimaftj Internal Revenue Service 
Naomi Verdugo^ Department of the Army 



Introduction 

The area of data problems is a vast ^'gold nilne" of discussion, but a vast "chasm 
of despair" when actually doing forecasts. As each participant explained his or her 
encounters with missing data, incorrect data, partial data, or multiple variations 
on each of these, it became evident that each lacked the kinds of data needed to do 
the job properly. Details of the discussions and son'e of the remedies for each of 
the three major categories of data problems are discussed below. 

Missing Data 

The problem of missing data became obvious from the first session as the 
dependence of agencies on each other's data was noted. For instance, if data 
collection for an item is suspended, then user agencies must adjust for this lack of 
data. Participants from the Veterans* Administration, the Bureau of the Census, and 
the Departments of Agriculture, Labor, Health and Human Services, and Education 
indicated that they experienced similar situations. The question arose, "What do we 
do about the lack of data?** 

The most frequent remedy chosen was the use of similar data, modified with the 
advice of a number of experts. First, the forecaster uses judgment to estimate 
missing data. Then that guess is given to others who are thought to be experts in 
that area for their approval. Finally, the guess is verified over time and modified 
in future years to adjust for expert error. 

Incorrect Data 

The revelation by forecasters that vast shifts in data often occur, led to a 
lively discussion on the reliance other forecasters place on data that are often 
incorrect. "The man who has one watch knows what time it is," was the phrase used 
to describe a forecaster*s normal methodology. That is, with data in hand from one 
respected source, the forecaster confidently predicts the future. He "knows" the 
past. Sometimes the forecaster is shocked to find that the past has changed, 
invalidating the forecast work. When the "watch" of history, the foundation of the 
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work, changes slightly, the forecast is weak or even destroyed. The question 
becomes, "What can wc do about this?" 

The answers to this question were less tangible. It would have been easy for 
such a discussion to break down into an accusatory session, with one group blaming 
another for such data errors that lead to forecast mistakes. Therefore, the 
discussion leaders directed this phase of the discussion to the more positive areas 
of 1) how we notify people of the caveats on our forecasts and 2) how wc adjust 
rapidly to changes in other agency trend data. 

In the area of caveats, all agreed that users do not read them. In general, 
people want a number, not an explanation. Some forecasters, noting horrible 
experiences in the past with this, indicated they will not provide information over 
the telephone. They insist the user get the data in writing. Thus, the provider is 
assured that the user receives background explanation information with the 
forecast Others indicated that all tables they produce have confidence limits, and 
that information provided over the telephone also contains such limits. Therefore, 
if users choose to ignore the confidence limits, they do so "at their own risk." 

As far as adjusting rapidly to other agency data changes, the groups all 
indicated that the easiest way to do that is to understand how an agency prepares 
its data. Certain modeling techniques may be adjusted more rapidly than others, but 
no forecasting operation can change printed materials or information given prior to 
the development of adjusted forecasts. Thus, increased understanding of each 
other's methodologies and data shortcomings are mandatory. How do we better 
understand each other's situation? All agreed that having better 
communication— specifically, having more such conferences and meetings^is the key. 
With the level of interdependence expressed by all parties, it was surprising that 
individuals who had routinely used each other's data met, for the first time, at 
this conference. 

A type of data change that frequently occurs is the change in definition over 
time. This problem was discussed at great length. One aberration of this problem 
is the change, not of definition of a data item, but of the circumstances through 
which the data item was collected. A number of participants indicated that data 
appear to change when the position of a question on a questionnaire changes. Again, 
the only way to react positively to such problems is increased communication among 
forecasters. If users of data know in advance that a data item is either changing 
in definition or the collection procedures are being modified, then such concerns 
should be communicated. Many individuals indicated a willingness to consider user 
concerns with data collection modifications, if these concerns are known in advance. 
Such changes in data definition and collection lead to the final type of problem 
discussed— partial data. 

Partial Data 

Many forecasters indicated that models that have been developed and tested to be 
sufficiently accurate over time have been effectively destroyed by another .agency's 
change in data. This leaves the model with only part of the data it needs. This 
problem is especially prevalent when data are provided by other agencies, such as 
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States. If only 48 of the 50 States provide information, the obvious question is, 
"Can this data be used?" 

Reacting to this problem, the forecasters provided examples of forecasting to 
impute for missing entries in basic data. Thus, any forecasts using such basic data 
are "forecasts on forecasts " Therefore, the data problems described here may be 
self-perpetuating. If these data are not used because of lack of faith in the 
imputations, then there is insufficient or no data. In any case, circumstances are 
less than desirable at the outset of a forecasting task. 

The consensus of the group was that data problems are universal and that more 
discussion is the key to resolving them. 
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Topic B: FORECASTING METHODOLOGY 



Moderator: 



Paul J. Horn, National Center for Education Statistics 



Discussion Leaders: 



Gregory Spencer, Bureau of the Census 
Norman Saunders, Bureau of Labor Statistics 
Stuart Bernstein, Bureau of Health Professions 



Introduction 

Participants were eager to share information about problems they encountered in 
producing forecasts. All who commented said they thought the conference was an 
excellent idea and hoped that further conferences would be held. Besides discussing 
specific methods of producing forecasts or projections, the participants also talked 
about simplicity versus complexity; the role judgment plays in forecasting; how to 
convey confidence in the model; political or policy influences on the forecasting 
process; and model implementation on coinputers. 



Simplicity versus Complexity 

Methods discussed were decomposition, time series (smoothing, moving averages, 
extrapolation), ARIMA, Box-Jenkins, regression, input/output, and judgment Simpler 
methods were emphasized over the more complicated ones. Many forecasters said that 
their agencies had recently abandoned a coniplicated model for a simple one. It was 
generally agreed that complexity in modelling is not a virtue. At least one 
participant characterized very complex models as black box models, where only the 
inputs and outputs are known, but where virtually nothing is known about the process 
that produces the forecasts. Others said that complex models are not as reliable 
and are harder to replicate, especially where components are adiusted by expert 
opinion. While simple models are viewed as being generally more accurate, less 
confusing, and easier to explain, it was pointed out that they quite often miss 
significant turning points in the data. 

Disaggregation was also mentioned as leading to more complexity in the models. 
Highly di^^aggregated data tend to be highly variable, as well. Several forecasters 
said that while they do disaggregated forecasts, they would not want to show 
confidence intervals around them. While aggregated statistics are easier to deal 
with, often the forecaster must use the disaggregated data due to policymakers' 
concerns. 

Predicting turning points was considered to be one of the major difficulties in 
using any method. Since the past is all forecasters have to work with, any models 
they produce will miss sudden changes which cannot be modelled. One-time changes. 
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such as major policy changes, new legislation, and court rulings are especially 
difficult to deal with. Expert judgment, backed by good, solid research, is felt to 
be most effective in dealing with these types of change. 

Judgment 

Expert judgment in one form or another appears to be rather widely used in 
Federal forecasting. Several agencies produce their forecasts using only judgmental 
methods. While it was felt that judgmental models have their uses, it was pointed 
out that they have serious disadvantages. Among the difficulties mentioned were 
difficulty in understanding the process that leads to a particular forecast and a 
perceived greater vulnerability to lawsuits. Several agencies are moving away from 
judgmental models because of this. Other participants pointed out that judgment 
plays a role even in the more statistical procedures. However, the more 
mathematical and statistical techniques were felt to be less of a problem in this 
way. 



Confidence Limits 

The discussion of confidence limits provided lively interchanges in every 
session. Forecasters seem to be divided between those who say that confidence 
limits are not needed since their users do not care about them, and those who say 
that users must be given some measure of confidence, whether they want it or not. 
Several speakers said that policymakers simply want a number, that they are not 
interested in upper and lower limits. Others felt that it is the duty of 
forecasters to educate their principle users about the meaning of confidence 
limits. 

One participant observed that if there are significant policy implications and 
the high and low bounds are not presented, then a problem exists. An example of 
this is where the forecasts are used to decide whether or not to build a new sewage 
plant or school, for example. A simple point estimate might indicate that such a 
new facility would not be needed, where the interval forecast might indicate the 
opposite. 



Politics and Policy 

At several points, the implications of politics and policy on forecasting were 
discussed. Political and policy concerns can influence methodology in several 
ways. First, a method might be chosen to avoid the negative consequences of some 
other model. As one participant put it, "You won't get sued for extrapolation." 
Second, policymakers might dictate what method to use by the questions they ask. 
They are not interested solely in what is going to happen. They also want to know 
what might influence what will happen. Third, a change in the law or interpretation 
of policy might change the process that is being mode.;d. For example, the number 
of people participating in a program might drastically increase or decrease as a 
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result of alternatives to the formulas for eligibility. The consequences of such 
change might be impossible to model due to lack of data. 



Computer Resources 

Discussions of software used to forecast also generated a great amount of 
interest. Several agencies have developed models for the personal computer using 
spreadsheets. Other agencies use commercial statistical packages both on mainframe 
computers and on personal computers. Several speakers thought that it would be a 
good idea to have a clearinghouse for sharing ,modsls or software. Other resources 
discussed were electronic bulletin boards and user groups. Some agencies have tried 
contracting to produce models for forecasting, but have found the experience less 
than satisfying. The main reason mentioned for seeking outside help was lack of 
statisticians and economists on their staffs. 
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Topic C: FORECAST EVALUATION 



Moderators: 



Debra Gerald, National Center for Education Statistics 
William Hussar^ National Center for Education Statistics 



Discussion Leaders: 



Howard Fullerton, Bureau of Labor Statistics 
Eileen Collins, National Science Foundation 
Ben Klugh, Department of Agriculture 



Introduction 

The need to evaluate the performance of different forecasting models is 
essential In this session, participants shared information on model evaluation, 
methods to evaluate models and forecasts, and efforts to improve forecast accuracy 
in their agencies. Details of the discussions are presented below. 

Model Evaluation Activity 

There was a wide range of responses concerning the frequency of model 
evaluation. While many of the forecasters routinely evaluate their models and 
short-term results, fev/ have done formal evaluation studies due to long forecast 
horizons or the lack of final data. A question on the frequency of formal 
evaluations yielded a range of responses from no formal review, to official reviews 
every few years, to those conducted annually. While a few forecasters had several 
years of data for evaluating model results, others had to evaluate models they had 
recently developed. 

Model evaluations were found to serve a number of purposes. Some forecasters 
cited that the evaluations often cause further reconsideration of the model. Others 
noted that they cause a reexamination of the data. 



Evaluation Methods 

The most common measures of forecast accuracy used are Root Mean Square Error 
(RMSE) and Mean Absolute Percentage Error (MAPE). Other forecasters use Theil 
Decomposition to help improve forecast accuracy. 

Some forecasters were concerned with methods to calculate errors. While these 
participants were more concerned with measurement and nonsampling errors, one 
forecaster noted that boot:;trapplng is used to examine errors. 
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Some of the participants compare their forecasts to others. Changes arc made if 
there is a good reason. The participants generally agreed that comparing 
projections with the actuals may not always be the best way to measure accuracy. 
Rather, projections should be evaluated based on the conditions at the time of the 
forecast. 



Improving Accuracy 

The participants discussed problems that arise when several interrelated 
variables are being projected in the same agency. When several people are involved 
in forecasting individual components, lengthy discussions are held to reach a 
consensus on the final results. When forecasting both natior md subnational 
variables, forecasters have to decide at which level it is most important to achieve 
a high degree of accuracy. 

Some of the participants said they find it necessary to adjust results from 
models to account for people*s behavior. One forecaster cited a study which shows 
that what people say they will do is a poor predictor of what they will actually do, 
for example. 
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FEDERAL FORECASTERS CONFERENCE 



Sponsored by the National Center for EducatJoti Statistics 
Quality Inn Capitol Hill Hotel 
April 25, 1988 



TIME ACTIVITY 



9:00-9:30 Registration 
9:30-10:00 Welcoming Rcmmrks 

William Sonnenberg 

Statistician, National Center for Education Statistics 
Dr. Bruno Manno 

Chief of Staff, Office of Educational Research and Improvement 
Dr. Charles D. Cowan 

Chief Statistician, National Center for Education Statistics 



Three (3) concurrent work sessions will run on different forecasting issues. Each 
session will accommodate 10-20 individuals. These are scheduled as follows: 



SESSION 1 


Topic A 


Topic 3 


Topic C 


10:15-11:15 


bata Problems 
(What are they and 
how do you resolve 
them?) 


Forecasting 
Methods 

(What types of 
models are used?) 


Forecast 
Evaluation 

(What are you doing 
to improve forecast 
accuracy?) 




Discussion Leader: 
Ted Van Hintum 
Veterans* 
Administration 


Discussion Leader: 
Dr. Gregory Spencer 
Bureau of the 
Census 


T>iscussion Leader: 
Howard Fullerton 
Bureau of Labor 
Statistics 




NCES Moderator: 
W. Sonnenberg 


NCES Moderator: 
Paul J. Horn 


NCES Moderator: 
Debra Gerald 


SESSION 2 
11:30-12:30 


Discussion Leader: 
Russ Geiman 
Internal Revenue 
Service 


Discussion Leader: 
Norman Saunders 
Bureau of Labor 
Statistics 


Discussion Leader: 
Dr. Eileen Collins 
National Science 
Foundation 




NCES Moderator: 
W, Sonnenberg 


NCES Moderator: 
Paul J. Horn 


NCES Moderator: 
Dr. William Hussar 
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2:00-3:30 Feature Presentation 

"Forecasting: Problems and Solutions" 

Dr. !• Scott Armstrong 

Wharton School, University of Pennsylvania 

Director, International Institute of Forecasters 

Editor, International Journal of Forecasting 

Demonstration 

"The Time Machine" Forecasting Software Package 
Fred Collopy 

Wharton School, Universit-;' of Pennsylvania 



SESSION 3 
3:45-4:45 



Topic A 

Discussion Leader: 
Dr. N. Verdugo 
Department of the 
Army 

NCES Moderator: 
W. Sonnenberg 



Topic B 

Discussion Leader* 
Stuart' Bernstein 
Bureau of Health 
Professions 

NCES Moderator: 
Paul J. Horn 



Topic C 

Discussion Leader: 
Ben Klugh 
Department of 
Agriculture 

NCES Moderator: 
Debra Gerald 



4:45-5:00 Closing Remarks 

William Sonnenberg 



Copies of attending agencies* forecasting publications are on display in the exhibition 
area. 
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LIST OF PARTICIPANTS 



Tullio Albcrtini 

Bureau of Health Professions 



Allan Eck 

Bureau of Labor Statistics 



Trevor A ^eyne 
Congressional Budget Office 



Edward Flynn 

Energy Information Administration 



Da reel M Battle 

Bureau of Labor Statistics 



Howard Fullerton 
Bureau of Labor Statistics 



William Bell 
Bureau, of the Census 



Janice Fumanti 

Bureau of Labor Statistics 



Stuart Bernstein 

Bureau of Health Professions 



Russ Geiman 

Internal Revenue Service 



Douglas Braddock 
Bureau of Labor Statistics 



Debra Gerald 

National Center for Education Statistics 



Jonathan Budd 

National Institute of Justice 



Daniel Hecker 

Bureau of Labor Statistics 



William Buffington 
Veterans* Administration 



Lynne Heltman 
Veterans' Administration 



!Edwin Cissel 

Foreign Agriculture Service 



James Hogan 

Veterans* Administration 



Joel Cohen 

U.S. Postal Service 



Paul J. Horn 

National Center for Education Statistics 



Eileen Collins 

National Science Foundation 



William Hussar 

National Center for Education Statistics 



James Cultice 

Bureau of Health Professions 



Kenneth Johnson 

Bureau of Economic Analysis 



Mary Jane Curran 
Bureau of Labor Statistics 



Carl Kannapel 

Navy Recruiting Command 



John Dickstein 

Social Security Administration 



Bill Kelley 

Social Security Administration 



John Drabek 

Bureau of Health Professions 



Ben Klugh 

U.S. Department of Agriculture 
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John Kort 

Bureau of Economic Analysis 

Chester Levine 

Bureau of Labor Statistics 

John F. Long 
Bureau of the Census 

Esperanza Lopez 

Social Security Administration 

William Losaw 

Bureau of Health Professions 

Steven McLaughlin 
Veterans* Administration 

Ralph M. Monaco 

Economic Research Service, USDA 

Luda Murphy 

Bureau of Labor Statistics 

Rod Paschal 

Foreign Agriculture Service 

Valerie Personick 
Bureau of Labor Statistics 

Norman Saunders 
Bureau of Labor Statistics 



William Sonnenberg 

National Center for Education Statistics 

Gregory Spencer 
Bureau of the Cer ^us 

William Spencer 

U.S. Department of Agriculture 

Marge Stanton 

U.S. Department of Agriculture 
Betty Su 

Bureau of Labor Statistics 

Ted Van Hintum 
Veterans* Administration 

Naomi Verdugo 
Department of the Army 

Hal Wallach 

General Accounting Office 

Signe Wetrogan 
Bureau of the Census 

Chung Yeh 

U.S* Department of Agriculture 
Meyer Zitter 

National Academy of Sciences 
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FORECASTING PUBLICATIONS OF THE U.S. GOVERNMENT 



U.S. House of Representatives, Communication from the Board of Trustees, the 
Ferleral and Survivors Insurance and Disability Insurance Trust Funds to the 
Committee on Ways and Means, 1987 Annual Report of the Board of Trustees of the 
Federal Old-Age and Survivors Insurance and Disability Insurance Trust Funds^ 
March 1987. 

National Science Foundation, Projected Response of the Science, Engineering, 
and Technical Labor Market to Defense and Nondefense Needs: 1982-87^ January 1984. 

National Science Foundation, Scientific, Technological, and International 
Affairs, Division of Science Resources Studies, Employment Studies Group Program 
Plan and Announcement^ October 1987. 



U.S. Department of Agriculture, Economic Research Service, Agriculture and Rural 
Economics Division, Agricultural Finance Situation and Outlook Report, March 
1987. 

U.S. Department of Agriculture, Economic Research Service, Foreign Agricultural 
Service, World Agricultural Supply and Demand Estimates, April 1988. 

U.S. Department of Agriculture, Foreign Agricultural Service, World 
Agricultural Production, April 1988. 

U.S. Department of Agriculture, Foreign Agricultural Service, IVorld Cotton 
Situation, April 1988. 

U.S. Department of Agriculture, Foreign Agricultural Service, World Grain 
Situation and Outlook, April 1988. 

U.S. Department of Agriculture, Foreign Agricultural Service, World Oilseed 
Situation and Market Highlights, April 1988. 

U.S. Department of Agriculture, Statistical Reporting Service, Research 
Division, Application of Path Coefficient Analysis to a Plant Simulation Model, 
January 1983. 

U.S. Department of Agriculture, Statistical Reporting Service, Statistical 
Research Division, Assessment of Fixed vs. Stepwise Forecast Models to Predict 
Number of Soybean Pods with Beans per Plant, August 1984. 

U.S. Department of Agriculture, Statistical Reporting Service, Statistical 
Research Division, Fixed vs. Stepwise Forecast Models to Predict Number of Pods 
with Beans per Soybean Plant in Southern States, March 1985. 

U.S. Department of Agriculture, Statistical Reporting Service, Statistical 
Research Division, Evaluation of Sorgy, a Sorghum Crop Simulation Model, June 
1985. 
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U.S. Department of Agriculture, Statistical Reporting Service, Statistical 
Research Division, Covariance Analysis of Soybean Objective Yield Maturity 
Categories 7, 8, and 9, November 1985. 

U,S, Department of Agriculture, Statistical Reporting Service, Statistical 
Research Division, An Evaluation of Two Feedback Versions of the Ceres-Maize 
Model, February 1986. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Statistical Research Division, Corn Objective Yield: An Empirical Evaluation of 
the Use of 3, 4, or 5 Years Data to Develop Forecast Equations, June 1986. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Research and Applications Division, A Commodity Weighted Estimator, January 
1988. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Research and Applications Division, Forecasting Grain Weight per Corn Ear on 
August 7, February 1988. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Agricultural Statistics Board, Agricultural Prices, August 1987. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Agricultural Statistics Board, Crop Production, August 1987. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Agricultural Statistics Board, Agricultural Prices, September 1987. 

U.S. Department of Agriculture, National Agricultural Statistics Service, 
Agricultural Statistics Board, Crop Production, September 1987. 

U.S. Army Research Institute for the Behavioral and Social Studies, Manpower and 
Personnel Policy Research Group, Manpower and Personnel Research Laboratory, 
Projections of the Male Youth Population and Enlistment Propensity by Army 
Recruiting Battalion, 1980-1995^ October 1987. 

U.S. Department of Commerce, Bureau of the Census, Current Population Reports, 
Series P-25, no. 952, Projections of the Population of the United States by Age, 
Sex, and Race: 1983 to 2080, May 1984. 

U.S. Department of Commerce, Bureau of the Census, Statistical Research 
Division, Time Series Analysis of Household Headship Proportions: 1959-1985, 
July 1986. 

U.S. Department of Commerce, Bureau of the Census, Current Population Reports, 
Series P-25, no. 995, Projections of the Hispanic Population: 1983 to 2080, 
November 1986. 
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U.S. Department of Commerce, Bureau of the Census, Statistical Research 
Division, Forecasting Age-Specific Fertility Using Principal Components, August 
1987. 

U.S. Department of Commerce, Bureau of Economic Analysis, 1985 OBERS BEA 
Regional Projections: State Projections to 2035, vol. 1, 1985. 

U.S. Department of Commerce, Bureau of Economic Analysis, 7955 OBERS BEA 
Regional Projections: Metropolitan Statistical Area Projections to 2035, vol. 2, 
19£5. 

U.S. Department of Commerce, Bureau of Economic Analysis, Regional Economic 
Analysis Division, Linking Regional Economic Models for Policy Analysis, July 
1984. 

U.S. Department of Commerce, Bureau of Economic Analysis, Regional Economic 
Analysis Division, Baseline Projections to 1992 for States, BEA Regions, and the 
United States, No\cmhcT 1987. 

U.S. Department of Education, National Center for Education Statistics, 
Projections of Education Statistics to 1992-93: Methodological Report with 
Detailed Projection Tables, July 1985. 

U.S. Department of Education, Center for Education Statistics, Trends in 
Education 1975-76 to 1995-96, March 1987. 

U.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Degrees Conferred (National), April 1988. 

U.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Higher Education Enrollment (National), April 1988. 

b.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Public Elementary /Secondary Enrollments, April 1988. 

U.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Public High School Graduates, April 1988. 

U.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Public Elementary/ Secondary Teachers, April 1988. 

U.S. Department of Education, Center for Education Statistics, Targeted 
Forecast: Public Elementary and Secondary Current Expenditures, April 1988. 

U.S. Department of Energy, Energy Information Administration, Office of Energy 
Markets and End Use, Average Regional Energy Price Projections to the Year 2010, 
February 1987. 




U.S. Department of Energy, Energy Information Administration, Office 
of Energy Markets and End Use, International Energy Outlook 1986: 
Projections to 2000, April 1987. 

U.S. Department of Energy, Energy Information Administration, Office 
of Energy Markets and End Use, Annual Energy Review 1986, May 1987. 

U.S. Department of Energy, Energy Information Administration, Office 
of Coal, Nuclear, Electric, and Alternative Fuels, Annual Outlook for 
US. Coal 1987, May 1987. 

U.S. Department of Energy, Energy Information Administration, Office 
of Coal, Nuclear, Electric, and Alternative Fuels, Annual Outlook for 
US. Electric Power 1987: Projections through 2000, May 1987. 

U.S. Department of Energy, Energy Information Administration, Office 
of Energy Markets and End Use, Short-Term Energy Outlook: Quarterly 
Projections, January 1988. 

U.S. Department of Energy, Energy Information Administration, Office 
of Energy Markets and End Use, Annual Energy Outlook 1987: With 
Projections to 2000, March 1988. 

U.S. Department of Health and Human Services, Social Security 
Administration, Office of the Actuary, Disabled-Worker Projections OASDI 
Cost Estimates 1984, November 1984. 

U.S. Department of Health and Human Services, Public Health Service, 
Health Resources and Services Administration, Fifth Report to the 
President and Congress on the Status of Health Personnel in the United 
States, March 1986. 

U.S. Department of Health and Human Services, Social Security 
Administration, Office of the Actuary, Economic Projections for OASDHI 
Cost and Income Estimates: 1986, January 1987. 

U.S. Department of Health and Human Services, Social Security 
Administration, Office of the Actuary, Social Security Area Population 
Projections: 1987, August 1987. 

U.S. Department of Health and Human Services, Social Security 
Administration; Office of the Actuary, OASDI Long-Range Beneficiary 
Projection: 1987, February 1988. 

U.S. Department of Justice, National Institute of Justice, Drug Use 
Forecasting, January 1988. 

U.S. Department of Labor, Bureau of Labor Statistics, Monthly Labor 
Review, vol. 1 10, no. 9, September 1987. 
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U.S. Department of Labor, Bureau of Labor Statistics, Projections 
2000, March 1988. 



U.S. Department of Labor, Bureau of Labor Statistics, Occupational 
Outlook Handbook: 1988-89 Edition, April 1988. 

U.S; Department of Treasury, Internal Revenue Service, Research 
Division, Trend Analyses and Related Statistics: United Stages, 
Districts, Regions, and Service Centers, 1987 Update, March .1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division, Projections: Information and Withholding Documents, United 
States and Service Center Areas, Calendar Years 1987-1993, April 1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division, Projections: Number of Returns to be Filed, United States, 
Fall Update, FY 1989 Budget Cycle, Fiscal Years 1987-1995, August 1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division-, Projections: Forms 1040, 1040A, and 1040EZ by Full-Paid, 
Other-Than-FulUPaid, and Refunds, United States and Service Centers, Fall 
Update, Calendar Years 1987-1990, September 1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division, Projections: Number of Returns to be Filed, United States, 
Regions, and Service Centers, Calendar Years 1987-1994, October 1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division, Projections: Examination Classes, Number of Examination Class 
Returns to be Filed by District: Individual Partnership, Corporation, and 
Estate, Calendar Year 1988, November 1987. 

U.S. Department of Treasury, Internal Revenue Service, Research 
Division, Projections: Numbers of Returns to be Filed, Districts, 
Calendar Years 1987-1994, December 1987. 

Veterans' Administration, Office of Information Management and 
Statistics, Statistical Policy and Research Service, New Projections of 
the Veteran Population: A Review of the Data, July 1986. 

Veterans' Administration, Vocational Rehabilitation and Educational 
Service, The Montgomery G.L Bill: An Overview of Future Prospects, 
July 1987. 
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